How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task

Controlling for P-value inflation in allele frequency change in experimental evolution and artificial selection experiments

acceptedVersio

Assessment of nerve involvement in the lumbar spine: agreement between magnetic resonance imaging, physical examination and pain drawing findings

<p>Abstract</p> <p>Background</p> <p>Detection of nerve involvement originating in the spine is a primary concern in the assessment of spine symptoms. Magnetic resonance imaging (MRI) has become the diagnostic method of choice for this detection. However, the agreement between MRI and other diagnostic methods for detecting nerve involvement has not been fully evaluated. The aim of this diagnostic study was to evaluate the agreement between nerve involvement visible in MRI and findings of nerve involvement detected in a structured physical examination and a simplified pain drawing.</p> <p>Methods</p> <p>Sixty-one consecutive patients referred for MRI of the lumbar spine were - without knowledge of MRI findings - assessed for nerve involvement with a simplified pain drawing and a structured physical examination. Agreement between findings was calculated as overall agreement, the p value for McNemar's exact test, specificity, sensitivity, and positive and negative predictive values.</p> <p>Results</p> <p>MRI-visible nerve involvement was significantly less common than, and showed weak agreement with, physical examination and pain drawing findings of nerve involvement in corresponding body segments. In spine segment L4-5, where most findings of nerve involvement were detected, the mean sensitivity of MRI-visible nerve involvement to a positive neurological test in the physical examination ranged from 16-37%. The mean specificity of MRI-visible nerve involvement in the same segment ranged from 61-77%. Positive and negative predictive values of MRI-visible nerve involvement in segment L4-5 ranged from 22-78% and 28-56% respectively.</p> <p>Conclusion</p> <p>In patients with long-standing nerve root symptoms referred for lumbar MRI, MRI-visible nerve involvement significantly underestimates the presence of nerve involvement detected by a physical examination and a pain drawing. A structured physical examination and a simplified pain drawing may reveal that many patients with "MRI-invisible" lumbar symptoms need treatment aimed at nerve involvement. Factors other than present MRI-visible nerve involvement may be responsible for findings of nerve involvement in the physical examination and the pain drawing.</p

The DREAM Dataset: Supporting a data-driven study of autism spectrum disorder and robot enhanced therapy

We present a dataset of behavioral data recorded from 61 children diagnosed with Autism Spectrum Disorder (ASD). The data was collected during a large-scale evaluation of Robot Enhanced Therapy (RET). The dataset covers over 3000 therapy sessions and more than 300 hours of therapy. Half of the children interacted with the social robot NAO supervised by a therapist. The other half, constituting a control group, interacted directly with a therapist. Both groups followed the Applied Behavior Analysis (ABA) protocol. Each session was recorded with three RGB cameras and two RGBD (Kinect) cameras, providing detailed information of children’s behavior during therapy. This public release of the dataset comprises body motion, head position and orientation, and eye gaze variables, all specified as 3D data in a joint frame of reference. In addition, metadata including participant age, gender, and autism diagnosis (ADOS) variables are included. We release this data with the hope of supporting further data-driven studies towards improved therapy methods as well as a better understanding of ASD in general.CC BY 4.0DREAM - Development of robot-enhanced therapy for children with autism spectrum disorders

The DREAM Dataset: Behavioural data from robot enhanced therapies for children with autism spectrum disorder

This dataset comprise behavioural data recorded from 61 children diagnosed with Autism Spectrum Disorders (ASD). The data was collected during a large-scale evaluation of Robot Enhanced Therapy (RET). The dataset covers over 3000 therapy sessions and more than 300 hours of therapy. Half of the children interacted with the social robot NAO supervised by a therapist. The other half, constituting a control group, interacted directly with a therapist. Both groups followed the Applied Behavior Analysis (ABA) protocol. Each session was recorded with three RGB cameras and two RGBD (Kinect) cameras, providing detailed information of children's behaviour during therapy. This public release of the dataset noes not include video recordings or other personal information. Instead, it comprises body motion, head position and orientation, and eye gaze variables, all specified as 3D data in a joint frame of reference. In addition, metadata including participant age, gender, and autism diagnosis (ADOS) variables are included. All data in this dataset is stored in JavaScript Object Notation (JSON) and can be downloaded here as DREAMdataset.zip. A much smaller archive comprising example data recorded from a single session is provided in DREAMdata-example.zip. The JSON format is specified in detail by the JSON Schema (dream.1.1.json) provided with this dataset. JSON data can be read using standard libraries in most programming languages. Basic instructions on how to load and plot the data using Python and Jupyter are available in DREAMdata-documentation.zip attached with this dataset. Please refer to https://github.com/dream2020/data for more details. The DREAM Dataset can be visualized using the DREAM Data Visualizer, an open source software available at https://github.com/dream2020/DREAM-data-visualizer. The DREAM RET System that was used for collecting this dataset is available at https://github.com/dream2020/DREAM.Denna databas omfattar beteendedata från 61 barn diagnostiserade med Autismspektrumtillstånd (AST). Insamlat data kommer från en storskalig studie på autismterapi med stöd av robotar. Databasen omfattar över 3000 sessioner från mer än 300 timmar terapi. Hälften av barnen interagerade med den sociala roboten NAO, övervakad av en terapeut. Den andra hälften, vilka utgjorde kontrollgrupp, interagerade direkt med en terapeut. Båda grupperna följde samma standardprotokoll för kognitiv beteendeterapi, Applied Behavior Analysis (ABA). Varje session spelades in med tre RGB-kameror och två RGBD kameror (Kinect) vilka analyserats med bildbehandlingstekniker för att identifiera barnets beteende under terapin. Den här publika versionen av databasen innehåller inget inspelat videomaterial eller andra personuppgifter, utan omfattar i stället anonymiserat data som beskriver barnets rörelser, huvudets position och orientering, samt ögonrörelser, alla angivna i ett gemensamt koordinatsystem. Vidare inkluderas metadata i form av barnets ålder, kön, och autismdiagnos (ADOS). All data i den här databasen är lagrad som JavaScript Object Notation (JSON) kan här laddas ned i form av DREAMdataset.zip. Ett mycket mindre arkiv med exempeldata från en enstaka session kan laddas ned separat i form av DREAMdata-example.zip. JSON-formatet finns specificerat i form av ett JSON-schema som också bifogas med denna databas. JSON kan läsas med hjälp av standardbibliotek i de flesta programspråk. Instruktioner för att läsa och visualisera datat med hjälp av Python och Jupyter bifogas i DREAMdata-documentation.zip. Vänligen besök https://github.com/dream2020/data för detaljer. Databasen kan också visualiseras med hjälp av DREAM Data Visualizer, en enkel mjukvara som finns tillgänglig i form av öppen källkod via https://github.com/dream2020/DREAM-data-visualizer. Det fullständiga systemet som användes för inspelning av denna databas finns också tillgänglig via https://github.com/dream2020/DREAM

Robotövningar : Igenkänning och återgivande av demonstrerat beteende

The work presented in this dissertation investigates techniques for robot Learning from Demonstration (LFD). LFD is a well established approach where the robot is to learn from a set of demonstrations. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations. The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers. In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed. The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior. One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as a hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors

A New Look at Habits using Simulation Theory

Habits as a form of behavior re-execution without explicit deliberation is discussed in terms of implicit anticipation, to be contrasted with explicit anticipation and mental simulation. Two hypotheses, addressing how habits and mental simulation may be implemented in the brain and to what degree they represent two modes brain function, are formulated. Arguments for and against the two hypotheses are discussed shortly, specifically addressing whether habits and mental simulation represent two distinct functions, or to what degree there may be intermediate forms of habit execution involving partial deliberation. A potential role of habits in memory consolidation is also hypnotized

Cognitive Perspectives on Robot Behavior

A growing body of research within the field of intelligent robotics argues for a view of intelligence drastically different from classical artificial intelligence and cognitive science. The holistic and embodied ideas expressed by this research promote the view that intelligence is an emergent phenomenon. Similar perspectives, where numerous interactions within the system lead to emergent properties and cognitive abilities beyond that of the individual parts, can be found within many scientific fields. With the goal of understanding how behavior may be represented in robots, the present review tries to grasp what this notion of emergence really means and compare it with a selection of theories developed for analysis of human cognition, including the extended mind, distributed cognition and situated action. These theories reveal a view of intelligence where common notions of objects, goals, language and reasoning have to be rethought. A view where behavior, as well as the agent as such, is defined by the observer rather than given by their nature. Structures in the environment emerge by interaction rather than recognized. In such a view, the fundamental question is how emergent systems appear and develop, and how they may be controlled